Video tracking of baseball players which identifies merged participants based on participant roles

ABSTRACT

Video frames of a baseball game are analyzed to determine a track for the participants in the game and to update a digital record of the game. The merging of participants in a video frame is resolved by associating the participants&#39; tracks before and/or after the merging with a most likely participant role, such as a player, coach or umpire role. The role of one merged participant can be used to deduce the role of the other merged participant. In this way, the digital record can be completed even for the merged period. The role of a participant can be based, e.g., on the location of the participant relative to a base, a coach&#39;s box region, a pitcher&#39;s mound, a dugout, or a fielding position, or by determining that a participant is running along a path to a base or performing some other movement.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is related to commonly-assigned U.S. patent applicationSer. No. 12/976,766, filed Dec. 22, 2010, published as US2012/0162435 onJun. 28, 2012 and issued as U.S. Pat. No. 8,659,663 on Feb. 25, 2014,titled “Video Tracking Of Baseball Players To Determine The Start AndEnd Of A Half-Inning” and incorporated herein by reference.

BACKGROUND

Baseball is one of the most popular sporting events. Televisionproducers seek to entertain and inform audiences by employing the latestproduction techniques. For example, multiple camera angles,high-definition video and specialized equipment are used. Moreover,additional information such as graphics, player statistics and otherinformation can be provided. A human scorekeeper typically observesevents of the game and provides a record. However, it would be desirableto provide automated techniques for detecting events in a game, and forproviding a digital record of the game.

SUMMARY

Automated techniques for detecting events in a game, and for providing adigital record of the game, are provided.

In one embodiment, the images of participants in a video frame aremerged during a time interval. To identify the participants, theirtracks before and/or after the merging can be analyzed and associatedwith a most likely participant role. The digital record can then beupdated for the time interval in which the participants are merged aswell as earlier and/or later time intervals. In this way, the digitalrecord is complete even during the merged period.

In one approach, a method is provided for tracking participants in abaseball game at a baseball park, where the baseball park includes aplaying field. The method includes analyzing video frames obtained fromone or more cameras, where the video frames provide images of theparticipants in the baseball park. The method further includes, based onthe analyzing: determining a merged track of at least first and secondparticipants in the baseball park, where the merged track indicatessuccessive locations of the at least first and second participants inthe baseball park during a merged interval in which the first and secondparticipants are at least partly merged, and determining respectivenon-merged tracks of the at least first and second participants in thebaseball park during a non-merged interval in which the first and secondparticipants are not merged.

The method further includes identifying a role of the first participantfrom among a plurality of roles in the baseball game which is consistentwith the respective non-merged track of the first participant. Themethod further includes, responsive to the identifying, updating adigital record of the baseball game to associate the first participantwith the merged track.

Corresponding methods, systems and computer- or processor-readablestorage devices for performing the methods provided herein are provided.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1A depicts a baseball park and equipment for obtaining videoimages.

FIG. 1B depicts further details of the processing facility and camerasof FIG. 1A.

FIG. 1C depicts a relationship between camera, image and worldcoordinate systems, for use by the processing facility of FIG. 1A indetecting objects in the baseball park.

FIG. 2A depicts regions of the baseball park of FIG. 1A includingregions on and off the playing field.

FIG. 2B depicts the regions of FIG. 2A.

FIG. 3A depicts tracks of participants in the baseball park of FIG. 1Awhich are indicative of the start of a half-inning of a baseball name.

FIG. 3B1 depicts tracks of participants in the baseball park of FIG. 1Awhich are indicative of the end of a half-inning of a baseball game.

FIG. 3B2 depicts a digital record for the locations of the left fielderin FIG. 3B1.

FIG. 3B3 depicts the locations of the left fielder in FIG. 3B1,consistent with the digital record of FIG. 3B2.

FIG. 4A depicts ball trajectories in the baseball park of FIG. 1A whichare indicative of the start of a half-inning of a baseball game.

FIG. 4B depicts a video image of one bodily position of a participantwhich is indicative of a pitcher.

FIG. 4C depicts a video image of another bodily position of aparticipant which is indicative of a pitcher.

FIG. 5A depicts example locations of participants in the baseball parkof FIG. 1A, indicating potential merging of participants.

FIG. 5B depicts a video image of a pitcher and a coach who are notmerged, from the viewpoint of the camera 500 of FIG. 5A.

FIG. 5C depicts a video image of a pitcher and a coach who are merged,from the viewpoint of the camera 500 of FIG. 5A.

FIG. 5D depicts an outline image of the video image of FIG. 5C.

FIG. 5E1 depicts a timeline of the participants in FIG. 5E2 indicatingmerged and non-merged time intervals, from the perspective of the camera500 of FIG. 5A.

FIG. 5E2 depicts a portion of the baseball park of FIG. 5A, showingtracks of a pitcher and a coach in a time interval.

FIG. 5E3 depicts a digital record for the locations of the pitcher inFIG. 5E2.

FIG. 5E4 depicts a digital record for the locations of the coach in FIG.5E2.

FIG. 6A depicts a video image of a first baseman and a base coach whoare not merged, from the viewpoint of the camera 510 of FIG. 5B.

FIG. 6B depicts a video image of a first baseman and a base coach whoare merged, from the viewpoint of the camera 510 of FIG. 5B.

FIG. 6C depicts an outline image of the video image of FIG. 6B.

FIG. 6D1 depicts a timeline of the participants in FIG. 6D2 indicatingmerged and non-merged time intervals, from the perspective of the camera510 of FIG. 5A.

FIG. 6D2 depicts a portion of the baseball park of FIG. 5A, showingtracks of a first baseman and a base coach in a time interval,corresponding to the timeline of FIG. 6D1.

FIG. 6D3 depicts a digital record for the first base coach in FIG. 6D2.

FIG. 6D4 depicts a digital record for the first baseman in FIG. 6D2.

FIG. 7A depicts further example locations of participants in thebaseball park of FIG. 1A, indicating potential merging of participants.

FIG. 7B1 depicts a timeline of the participants in FIG. 7B2 indicatingmerged and non-merged time intervals, from the perspective of the camera510 of FIG. 7A.

FIG. 7B2 depicts a portion of the baseball park of FIG. 7A, showingtracks of a first baseman and a first base runner in a time interval,corresponding to the timeline of FIG. 7B1.

FIG. 7B3 depicts a digital record for the first baseman in FIG. 7B2,from the perspective of camera 510.

FIG. 7B4 depicts a digital record for the first base runner in FIG. 7B2,from the perspective of camera 510.

FIG. 7C1 depicts a timeline of the participants in FIG. 7C2 indicatingmerged and non-merged time intervals, from the perspective of the camera700 of FIG. 7A.

FIG. 7C2 depicts a portion of the baseball park of FIG. 7A, showingtracks of a first baseman and a runner headed to first base in a timeinterval, corresponding to the timeline of FIG. 7C1.

FIG. 7C3 depicts a digital record for the first baseman in FIG. 7C2,from the perspective of camera 700.

FIG. 7C4 depicts a digital record for the first base runner in FIG. 7C2,from the perspective of camera 700.

FIG. 8A depicts a video image of a first baseman and a base runner whoare not merged, from the viewpoint of the camera 700 of FIG. 7A.

FIG. 8B depicts a video image of a first baseman and a runner headed tofirst base who are merged, from the viewpoint of the camera 700 of FIG.7A.

FIG. 8C depicts an outline image of the video image of FIG. 8B.

FIG. 9A depicts an example method for tracking baseball participantsfrom a video sequence, and updating a digital record.

FIG. 9B depicts an example of further details of step 906 of FIG. 9A,for use in determining the start of a half-inning in step 912 of FIG.9A.

FIG. 9C depicts an example of further details of step 908 of FIG. 9A,for use in determining the start of a half-inning in step 912 of FIG.9A.

FIG. 9D depicts an example of further details of step 910 of FIG. 9A,for use in determining the start of a half-inning in step 912 of FIG.9A.

FIG. 9E depicts an example of a process for determining the start of ahalf-inning.

FIG. 9F depicts an example of further details of step 906 of FIG. 9A,for use in determining the end of a half-inning in step 912 of FIG. 9A.

FIG. 9G depicts an example of further details of step 914 of FIG. 9A,for use in detecting merged participants.

FIG. 9H depicts the baseball park of FIG. 1A including straight andcurved base miming paths.

DETAILED DESCRIPTION

Automated techniques for detecting events in a game, and for providing adigital record of the game, are provided.

FIG. 1A depicts a baseball park and equipment for obtaining videoimages. While a baseball game is provided as example, the techniquesprovided herein can be applied to other sporting events as well whichinvolve opposing teams and a game ball. A baseball park 100 can be abaseball stadium or a smaller facility, and includes a baseball field102. A number of video cameras obtain video images of the game as ittranspires in the baseball park. The video cameras can be, e.g., specialpurpose machine vision cameras for tracking, or television videocameras. For example, two cameras located at different heights, e.g.,cameras 160 and 162, are depicted. Any number of cameras can be used. Inone approach, two to six cameras, capturing color or monochrome images,can be used. The cameras have different points of view of the baseballpark. A processing facility 164 receives and processed frames of videoimages from the cameras. In one approach, the processing facility 164 isa mobile facility such as a truck which is parked outside the baseballpark. The processing facility 164 can subsequently transmit the capturedimages and other information via an antenna 145, to another locationsuch as a television broadcast facility. In another approach, theprocessing facility can be remote from the baseball park 100. Or, theprocessing facility can be a permanent facility, neither mobile norremote, such as one which is inside the baseball park.

The baseball park includes a playing field 102 which can be arrangedaccording to standard rules of baseball in the US, as described in the“Official Baseball Rules” of Major League Baseball (MLB). A baseballgame can include game which is played according to these rules orsimilar rules. The boundaries of the playing field are the foul lines114 and 124, the outfield boundary 126 which may be a fence or wall, andthe semicircle 107 around home plate 104. Line 130 is the boundarybetween the outfield 138 and the infield. The infield includes asquare/diamond region (including two sides 120 and 116) between the fourbases 104, 112, 118 and 122. The infield also includes a curved region136 which is between the reuions 134 and 138. Also provided are: aleft-side batter's box 106, a right-side batter's box 108, a catcher'sbox 110, a first base coach's box 140, a third base coach's box 142, apitcher's mound 132, on-deck circles 144 and 146, and dugouts 150 and148.

Additional cameras can be used as well to increase the accuracy of thetracking. As mentioned, the cameras can be, e.g., special purposemachine vision cameras. Or, television broadcast cameras can be usedwhich capture thirty frames or sixty fields per second, in one possibleapproach. Such cameras sense visible light. The locations of objects inthe baseball park, including participants and the baseball, can bedescribed in terms of a world coordinate system, also known as a freespace coordinate system, which is fixed relative to the earth or otherenvironment of interest, in one approach. The world coordinate systemincludes orthogonal directions represented by a Yw axis, an Xw axis, anda Zw axis (not shown) which extends out of the page. An origin of theworld coordinate system is chosen to be at the edge of home plate 104,as an example.

Each camera can be provided with sensors which detect intrinsic andextrinsic parameters of the camera when these parameters are variable.Intrinsic parameters, such as focal length, lens distortion and zoomsetting represent characteristics of the camera design and settings, anddo not depend on the position and orientation of the camera in space.Extrinsic parameters, such as tilt or pan, depend on the position andorientation of the camera in space. Such sensors can be provided usingtechniques known to those skilled in the art. For example, pan and tiltsensors can be attached to a tripod on which the camera is mounted. See,e.g., U.S. Pat. No. 5,912,700, issued Jun. 15, 1999, and incorporatedherein by reference. The sensors can be used to determine the field ofview of the camera, e.g., where the camera is pointing and what it cansee. Or, the cameras can be stationary and fixed so that they do notpan, tilt or zoom dynamically. In this case, broadcast cameras with apan-tilt-zoom (PTZ) capability could potentially be used to assist intracking.

It is also possible to determine camera extrinsic and intrinsicparameters without sensors, e.g., as described in Tsai's method. See,e.g., Tsai, Roger Y. (1986) “An Efficient and Accurate CameraCalibration Technique for 3D Machine Vision,” Proc. of IEEE Conf. onComputer Vision and Pattern Recognition, Miami Beach, Fla., 1986, pp.364-374. For example, one approach to determine the intrinsic andextrinsic parameters of a camera involves placing marks in variousmeasured or known locations in the event facility such that each marklooks different and at least one mark will always be visible to thecamera while the camera is pointed at the event facility. A computerusing optical recognition technology can find the mark in the videoframe and, based on the mark's size and position in the video frame,determine the camera parameters. Another approach to determiningintrinsic and extrinsic parameters of a camera involves placing marks invarious measured or known locations in the event facility such that eachmark looks different, but the marks may be removed after cameraparameters have been determined. A computer implementing a cameraparameter estimation algorithm based on manual user interaction ratherthan, or in addition to, image recognition can determine cameraparameters.

The processing facility 164 includes equipment which receives and storesthe captured images, time stamps the captured images, and processes thecaptured images to determine locations of objects such as participants(e.g., players and coaches) and the baseball, at successive points intime, e.g., based on the times at which the video frames were captured.When the captured images are provided in video signals from the cameras,the processing facility 164 can enhance the video signals based on thedetermined locations of the object, such as by displaying a graphicwhich depicts the locations. A series of successive locations of aparticipant or ball in the baseball park can be represented as a track.Generally, the locations of the participants in the baseball park can bedetermined by identifying pixels in the video frames which correspond tohuman figures, and determining a location in the baseball park based onlocations of the identified pixels in the video frames, and based onsensor and registration data of the one or more cameras. Statisticalinformation regarding the object can be obtained, including location,speed, direction, distance traveled, height, time in the air and soforth.

FIG. 1B depicts further details of the processing facility and camerasof FIG. 1A. The computer system 10 is a simplified representation of asystem which might be used at the processing facility 164 (FIG. 1A), forinstance. The computer system 10 includes a storage device 11 such as ahard disk or portable media, a network interface 12 for communicatingwith other computer systems, one or more processors 13 for executingsoftware instructions, a working memory 14 such as RAM for storing thesoftware instructions after they are loaded from the storage device 11,for instance, camera interfaces 15 and 17, and a user interface display16. The storage device 11 may be considered to be a processor readablestorage device having processor readable code embodied thereon forprogramming the processor 13 to perform methods for providing thefunctionality discussed herein. The user interface display 16 canprovide information to a human operator based on the data received fromthe cameras via the interfaces 15 and 17. The user interface display 16can use any known display scheme, whether graphical, tabular or thelike. In addition to an on-screen display, an output such as a hard copyfrom a printer can be provided to report results. Results can also bereported by storing data at the storage device 11 or other memory, e.g.,for later use. Results could also be sent via the network interface andthe Internet or other wide area network, to another, central storagelocation. The results can include a digital record of a baseball game.

An example camera 18 includes intrinsic parameter sensors 19 andextrinsic parameter sensors 20. The intrinsic parameter sensors 19 canidentify a zoom setting, whether an expander is used and so forth. Theextrinsic parameter sensors 20 can identify an orientation of thecamera, such as a pan and tilt of the camera. Note that sensors are notneeded when the parameter of concern is not changing. The camera 18communicates image data, whether analog or digital, in addition to datafrom the intrinsic parameter sensors 19 and the extrinsic parametersensors 20 to the computer system 10 via the camera interface 15.Similarly, another camera 21 with intrinsic parameter sensors 22 andextrinsic parameter sensors 23 can communicate data to the camerainterface 17. Data from more than two cameras can be received as well.

Further, the functionality described herein may be implemented using oneor more processor readable storage devices (e.g., 11 and 14) havingprocessor readable code embodied thereon for programming one or moreprocessors to perform the processes described herein. The processorreadable storage devices can include non-transitory, tangible computerreadable media such as volatile and nonvolatile media, removable andnon-removable media. Computer readable media includes volatile andnonvolatile, removable and non-removable media implemented in any methodor technology for storage of information such as computer readableinstructions, data structures, program modules or other data. Computerreadable media includes RAM, ROM, EEPROM, flash memory or other memorytechnology, CD-ROM, digital versatile disks (DVD) or other optical diskstorage, magnetic cassettes, magnetic tape, magnetic disk storage orother magnetic storage devices, or any other non-transitory, tangiblemedium which can be used to store the desired information and which canbe accessed by a computer.

FIG. 1C depicts a relationship between camera, image and worldcoordinate systems, for use by the processing facility of FIG. 1A indetecting objects in the baseball park. A camera coordinate system 36,which includes orthogonal axes X_(C), Y_(C) and Z_(C) in threedimensions, is fixed relative to the camera. The origin of thecoordinate system may be at the center of the lens, in one possibleapproach, modeling the camera as a pinhole camera. An image coordinatesystem 30, also referred to as pixel space, includes orthogonal axes Xand Y in two-dimensions, and is fixed relative to a captured image 32. Aworld coordinate system 37, which includes orthogonal axes X_(W), Y_(W)and Z_(W), is fixed relative to, e.g., the earth, a baseball park orother event site, or other reference point or location. Generally, it isdesirable to describe the position and/or path of the tracked object 31in the world coordinate system 37 as this is typically the coordinatesystem in which its motion is most relevant to the user, and allowseasier integration of the information from several cameras. The line ofposition 34 is an imaginary line which extends from the origin of thecamera coordinate system 36 through a pixel in the image 32,intersecting the pixel at a point 33, and through the tracked object 31.Each pixel in the image 32 corresponds to a different line of position.Point 33 in the captured image 32 represents the location of the object31 in the image. The location of the object in the image 32 can berepresented by coordinates (sx, sy) in a coordinate system which has itsorigin at a corner of the image, in one approach. The coordinates mayidentify the center of the object. When the object is a tracked humanparticipant, characteristics such as the outline of the participant canbe detected.

Further, the line of position can be represented by a 3-D vector (LOP)which has unity magnitude, in one approach. The vector can be defined bytwo points along the LOP. The vector can be represented in the worldcoordinate system 37 using an appropriate transformation from the imagecoordinate system. The Z_(C) axis of the camera coordinate system, whichis the optical axis 35 of the camera, intersects the captured image at apoint represented by coordinates (0_(x), 0_(y)). A two-dimensionalcoordinate system extending from (0_(x), 0_(y)) can also be defined.

The camera registration process involves obtaining one or moretransformation matrices which provide a conversion between the imagecoordinate system 30 and the world coordinate system 37. Furtherinformation can be found in E. Trucco and A. Verri, “Introductorytechniques for 3-D computer vision,” chapter 6, Prentice Hall, 1998,U.S. Pat. No. 5,912,700, issued Jun. 15, 1999, and U.S. Pat. No.6,133,946, issued Oct. 17, 2000, each of which is incorporated herein byreference.

FIG. 2A depicts regions of the baseball park of FIG. 1A includingregions on and off the playing field. By subdividing the baseball parkinto regions, the locations of the participants can be associated with aparticular region at a particular point in time in the digital record.Ideally, a record can be provided which identifies the location of eachparticipant throughout the game, where each location at time t, L(t), isprovided by coordinates such as world coordinate (Xw(t),Yw(t)). Further,each location can be mapped to one of the regions shown. In practice,the regions can be configured in different ways, and the exampleprovided is one possible implementation. Regions may overlap, since therange of positions of the participants may overlap. For example, thefirst baseman's region 212 can overlap with the first base runner'sregion 216, and both regions can include first base.

The outfield is divided into a left field region 200, a center fieldregion 202 and a right field region 204. A left-side out-of-bounds (orfoul territory) region 250 is bounded by the left-side out-of-playboundary 244, left-side foul line 124, and line 235, and excludes thedugout region 240. The region 250 could also exclude the third base linecoach's box region 224 and the left-side on-deck circle region 234. Aright-side out-of-bounds region 252 is bounded by the right-sideout-of-play boundary 246, right-side foul line 114, and line 237, andexcludes the dugout region 238. The region 252 could also exclude thefirst base line coach's box region 214 and the right-side on-deck circleregion 236. A central, out-of-bounds region 242 is bounded by lines 235and 237. Also provided are a catcher's box region 233, a left-sidebatter's box region 230, a right-side batter's box region 232, a firstbase runner's region 216, a first base region 218, a second base region220, a third base region 222, a first baseman's region 212, a secondbaseman's region 210, a shortstop's region 208, a third baseman's region206, and a pitcher's mound region 248. The dugout regions are separatefrom the playing field. These regions are defined around landmarks onthe playing field to determine if a participant is at the landmark. Forexample, a participant in the region 218 is assumed to be at first base.A landmark can include, e.g., a base, a pitcher's mound or an areamarked by chalk lines, such as an on-deck circle, coach's box, catcher'sbox or batter's box. Each region can have a specified size, and bepositioned in specified way with respect to an associated landmark.

FIG. 2B depicts the regions of FIG. 2A. Here, for greater clarity, onlythe regions are depicted.

FIG. 3A depicts tracks of participants in the baseball park of FIG. 1Awhich are indicative of the start of a half-inning of a baseball game.According to common baseball rules, a game is played between two teams,each composed of nine players, that take turns playing offense (hitting)and defense (fielding or pitching). A half-inning is a period in whichone team assumes the offense role and the other team assumes the defenserole, until three outs are recorded. A full inning includes twohalf-innings. The start of the first half-inning is the same as thestart of the full inning, and the end of the second half-inning is thesame as the end of the full inning. In providing a digital record of abaseball game, it is helpful to automatically detect the start or end ofa half-inning from video of the baseball park and to record thisoccurrence. Typically, a scorekeeper observes the start or end of ahalf-inning. An automated detection capability avoids the need to accessthe scorekeeper's system, and provides the ability to automaticallycross-reference to associated video segments.

At the start of a half-inning of a baseball game, the players on thefield which have been playing defense, return to the dugout region, andthe players in the dugout, which have been playing offense, move ontothe field. Typically, video images captured from the cameras showmultiple players concurrently jogging from the dugout region to theplaying field. Particular camera viewpoints can be especially useful indetecting this, such as an overhead viewpoint or other elevatedviewpoint which sees the entire playing field. Each participant isrepresented by a circle with a “P” inside. For example, the left fielder300 travels in a track/path 302, the center fielder 304 travels in atrack 306, the right fielder 308 travels in a track 310, the thirdbaseman 312 travels in a track 314, the shortstop 316 travels in a track318, the second baseman 320 travels in a track 322, the first baseman324 travels in a track 326, the pitcher 328 travels in a track 330 andthe catcher 336 travels in a track 338. Each participant of the otherteam, which has concluded playing defense, travels to the dugout region238, for instance. The first and third baseline coaches (not depicted)can also travel to the respective coach's boxes.

Generally, the digital record of the baseball game can be updated toindicate that the start of the half-inning is about to occur, if thetracks indicate that the start of the half-inning is about to occur, orto indicate that the end of the half-inning has occurred, if the tracksindicate that the end of the half-inning is about to occur. In oneapproach, the start of the half-inning can be defined as some timeinterval such as a few seconds before the first game pitch of thehalf-inning, i.e., the first pitch with the batter in the batter's box.Warm up pitching time is not counted as part of the half-inning, but itis a very good indication that the start of the half-inning is about tooccur. An indication that the start of the half-inning is about to occurcan be the detection of one or more events which are known to occurbetween successive half-innings, based on rules, traditions and othernorms of the game, and which are associated with the start of ahalf-inning. These are events which are known to occur betweensuccessive half-innings but not typically during a half-inning.

The digital record can thus be updated to indicate that the start of thehalf-inning has actually occurred after it is updated to indicate thatthe start of the half-inning is about to occur. The updating of thedigital record to indicate that the start of the half-inning hasoccurred can be based on detecting an event which is known to occur mostoften or always during a half-inning. One example is detecting a firstthrown pitch when a batter is in a batter's box of the baseball park,after the tracks indicate that the start of the half-inning is about tooccur. The detecting the first thrown pitch can include determining abodily movement of one of the participants (e.g., the pitcher) based onthe video frames, and determining that the bodily movement correspondsto a pitching motion. The detection of a thrown pitch can also be basedon tracking the ball as it moves from the pitcher's mound to home plate.

In case the first thrown pitch is not detected, a later event such as alater thrown pitch, the batter swinging, or the batter running towardfirst base can indicate that the start of the half-inning has occurred.Thus, the combination of detecting an event (an inter-half-inning event)which is known to occur most often or always between half-inningsfollowed by detecting another event (an intra-half-inning event) whichis known to occur most often or always during a half-inning can be usedto signal that a half-inning has started. Note that detecting an eventwhich is known to occur most often or always during a half-inning maynot be sufficient to determine that a half-inning has started (withoutdetecting an event which is known to occur most often or always betweenhalf-innings) because many of these events will be detected.

In one approach, a condition is imposed in which the start of ahalf-inning is not found unless the intra-half-inning event follows theinter-half-inning event within a specified time interval such as severalminutes.

In one approach, the end of the half-inning can be defined as when thelast out is made in the half-inning. In contrast to the case ofdetecting the start of a half-inning using a combination of two events,the detection of one event which is known to most often or always occursduring a half-inning can be used to signal that a half-inning has ended.Note that the same detected one or more events can signal that both anend of a half-inning of the baseball game has occurred, and that a startof the next half-inning of the baseball game is about to occur. Or, afirst event can signal that an end of a half-inning of the baseball gamehas occurred, and a subsequent second event can signal that a start ofthe next half-inning of the baseball game is about to occur.

Examples of events which are known to occur most often or always duringa half-inning follow. With one criterion, the tracks indicate that thestart of the half-inning is about to occur, e.g., is imminent, when thetracks of at least an integer number N>2 of the participants indicatethat the at least N participants have moved from the dugout region tothe playing field within a specified time interval. As more participantsare detected moving from the dugout region to the playing field, thereis a higher probability that the start of the inning is about to occur.

With another criterion, the tracks indicate that the start of thehalf-inning is about to occur when the tracks of the at least Nparticipants indicate that the at least N participants have reached atleast a specified minimum speed when moving from the dugout region tothe playing field. Typically, the participants jog or run onto the fieldto move to their respective fielding positions, and the pitcher moves tothe pitcher's mound. By obtaining a track which indicates aparticipant's locations at successive points in time, the instantaneousspeed of the participant at the different time points can be determined,and a determination can be made that the participant is jogging.

With another criterion, the tracks indicate that the start of thehalf-inning is about to occur when the tracks of at least an integernumber N>2 of the participants indicate that the at least N participantshave moved from the dugout region to the outfield within a specifiedtime interval.

With another criterion, the tracks indicate that the start of thehalf-inning is about to occur when at least one of the tracks indicatesthat one of the participants has moved to a coach's box region of thebaseball park from a dugout region of the baseball park. This would bethe coach of the team which assumes the offense role in the nexthalf-inning.

Multiple criteria can be used to obtain a probability that the start ofthe half-inning is about to occur. For example, a higher probability canbe associated with detecting the participants moving from the dugoutregion to respective regions in the playing field which are fieldinglocations, and remaining at those locations for a minimum amount oftime. The probability can be in proportion to the amount of time at thefielding location. A fielding location can be the region of the baseballfield in which a player is assigned. For example, region 212 in FIG. 2Bis the fielding location of the first baseman, also referred to as thefirst baseman's region. A higher probability can be associated withdetecting a greater number of participants moving from the dugout regionto the field. A decision as to whether the start of the half-inning isabout to occur can be made based on whether the probability exceeds athreshold. The threshold can be established by processing video datafrom multiple games.

In some case, the participants move from the dugout region to theleft-side out-of-bounds region 250, such as to congratulate a player whois running from third base to home plate to score a run. However, thiscan be distinguished from the start of a half-inning because theparticipants do not move onto the playing field and, in fact, areforbidden from moving onto the playing field. In another case, theparticipants move from the dugout region to the infield square region134 such as when there is a melee. However, this can be distinguishedfrom the start of a half-inning because the participants do not moveonto the playing field and remain in respective fielding positions. Nordo they moving to fielding positions in the outfield. In another case, acoach moves from the dugout region to the pitcher's mound to speak withthe pitcher. However, this can be distinguished from the start of ahalf-inning because the coach does not move to a fielding position, nordo multiple participants move to the field, such as to fieldingpositions. In another case, a participant moves from the dugout regionto the on-deck circle region. However, this can be distinguished fromthe start of a half-inning because the participant does not move ontothe playing field.

FIG. 3B1 depicts tracks of participants in the baseball park of FIG. 1Awhich are indicative of the end of a half-inning of a baseball game. Theend of a half-inning can be detected, e.g., by using the reverse of thecriteria associated with the start of a half-inning. At the end of ahalf-inning of a baseball game, the players on the field which have beenplaying defense, return to the dugout region. Typically, video imagescaptured from the cameras show multiple players concurrently joggingfrom the playing field to the dugout region. For example, the leftfielder 300 travels in a track 342, the center fielder 304 travels in atrack 348, the right fielder travels in a track 354, the third baseman312 travels in a track 344, the shortstop 316 travels in a track 346 thesecond baseman 320 travels in a track 350, the first baseman 324 travelsin a track 356, the pitcher 328 travels in a track 352 and the catcher336 travels in a track 358. Each participant of the other team, whichhas concluded playing defense travels to the dugout region 238, forinstance. The first baseline coach 332 travels in a track 360 back tothe other team's respective dugout region 238. The third baseline coach(not depicted) can also travel back to the other team's respectivedugout region 238.

With one criterion, the tracks indicate that the end of the half-inninghas occurred when at least one of the tracks indicates that one of theparticipants has moved to a coach's box region of the baseball park froma dugout region of the baseball park. This would be the coach of theteam which assumes the offense role in the next half-inning. Withanother criterion, the tracks indicate that the end of the half-inninghas occurred when the tracks of at least an integer number N>2 of theparticipants indicate that the at least N participants have moved to thedugout region from the playing field within a specified time interval.With another criterion, the tracks indicate that the end of thehalf-inning has occurred when the tracks of the at least N participantsindicate that the at least N participants have reached at least aspecified minimum speed when moving to the dugout region from theplaying field. With another criterion, the tracks indicate that the endof the half-inning has occurred when at least one of the tracksindicates that one of the participants has moved from a coach's boxregion of the baseball park to a dugout region of the baseball park.This would be the coach of the team which concludes the offense role.

As before, multiple criteria can be used to obtain a probability thatthe end of the half-inning has occurred. For example, a higherprobability can be associated with detecting a greater number ofparticipants moving to the dugout region from the field.

FIG. 3B2 depicts a digital record for the locations of the left fielderin FIG. 3B1. FIG. 3B3 depicts the locations of the left fielder in FIG.3B1, consistent with the digital record of FIG. 3B2.

As an example, the track 342 of the left fielder 300 is depicted atdifferent time points t0-t5. In practice, many more time points can beobtained, e.g., several per second. In one approach, the location of aparticipant is determined for each video frame. The digital record isstored in a storage media, such as the storage media 11 in FIG. 1B. Thedigital record can be in the form of a database or table which includesan entry for each time point. A digital record can be provided for eachparticipant. In some cases, a digital record can be provided for eachcamera which is used to detect a particular participant. A particularparticipant can be identified by his or her role or position in thebaseball game. For example, for the team playing defense, there are nineavailable player roles. e.g., pitcher, catcher, first baseman, secondbaseman, shortstop, third baseman, left fielder, center fielder andright fielder.

For the team playing offense, there are player roles such as batter,base runners and on-deck hitter, and coach roles, such as first basecoach and third base coach. For each team there are also coaches such asa head coach/manager, batting coach, bench coach, pitching coach andbullpen coach. Major League Baseball restricts the number of uniformedstaff to six coaches and one manager during the course of a game.Further, participants can include umpire roles such as the home plateumpire, the base umpires (first base umpire, second base umpire, thirdbase umpire) and two additional umpires which may be present duringplayoffs and other special games and are typically stationed along theoutfield foul lines as the left-field and right-field umpires (asoutfield umpires).

A player role could also be classified more generally such as anoutfielder role (left fielder, center fielder or right fielder), or aninfielder role (first baseman, second baseman, shortstop or thirdbaseman).

A particular participant could also be identified by his or her name,jersey number or other identifier. Each entry can identify the time, thelocation of the participant (in world coordinates) and the correspondingregion of the baseball park. The location coordinates provided can haveany desired units. The magnitudes of the example location coordinateprovided herein were measured based on actual locations in the drawings.

The time point t0 is the first time point in the track 342, and thefinal time point (tf) in the track is t5. Note that the locationsassociated with time points t0-t2 are in the region 200, the locationsassociated with time points t3 and t4 are in the region 250, and thelocation associated with time point t5 is in the region 240. The speedof the participant at the location of t2, for instance, can bedetermined from the distance between (Xw(t2),Yw(t2) and Xw(t1),Yw(t1),divided by the time increment t2-t1. A similar analysis can be made foreach of the other participants.

FIG. 4A depicts ball trajectories in the baseball park of FIG. 1A whichare indicative of the start of a half-inning of a baseball game. Asmentioned, the start of a half-inning can be detected based on variouscriteria which involve movement of the participants from the dugoutregion to the playing field. Another criterion involves determining oneor more trajectories of a ball in the baseball park based on the videoframes, and determining whether the start of the half-inning is about tooccur based on the trajectories. For example, the trajectories canindicate that the start of the half-inning is about to occur when thetrajectories extend from one participant to another participant, andback from the another participant to the one participant, indicating theparticipants are fielders warming up by playing catch. The trajectoriescan extend above the playing field, in the region outside the infieldsquare. The infield square is the region bounded by the four bases. Theregion outside the infield square could include the regions 136 and 138,for instance, in FIG. 4A, or the corresponding subdivided regions suchas in FIG. 3B3.

For example, trajectories 400 and 402 indicate that the left fielder 300and the center fielder 304 are playing catch. Trajectories 404 and 406indicate that the shortstop 316 and the second baseman 320 are playingcatch. Detection of such activity is indicative of the start of ahalf-inning, since the players often warm up briefly in this manner.Moreover, the fact that several balls are in motion at the same time indifferent parts of the field is especially indicative of the onset ofthe start of a half-inning. Moreover, trajectory 408 denotes a warm-uppitch from the pitcher 328 to the catcher 336, and trajectory 410denotes a return throw to the pitcher from the catcher. Under thebaseball rules, at the start of a half-inning, an existing pitcher ispermitted five warm-up pitches and a new pitcher is permitted eightwarm-up pitches. Detection of such activity indicates that the start ofa half-inning is about to occur, since the pitcher almost always warmups in this manner. A determination that the start of a half-inning isabout to occur can also involve a determination that no participant isin the batter's box regions 230 and 232, since warm up pitches are takenwith no batter present. When the start of the half-inning does occur,the batter is present in the batter's box. Thus, warm up pitches with nobatter present followed by the batter entering the batter's box regionis a cue that the start of the half-inning has occurred.

A participant 412 of the team playing offense warms up in the on-deckcircle 146.

Alternatively, or additionally, the fact that a pitcher has thrown apitch can be detected by using the video frames to analyze the bodilymovement of the participant who is located at the pitcher's mound. Inthis case, a determination of whether the start of the half-inning isabout to occur can be based on the bodily movement, where the bodilymovement indicates that the start of the half-inning of the baseball isabout to occur when the bodily movement corresponds to a pitchingmotion, while no batter is detected in a batter's box region of theplaying field, indicating that the participant is taking a warm uppitch.

For example, FIG. 4B depicts a video image of one bodily position 420 ofa participant on a pitcher's mound 422 which is indicative of a pitcher.FIG. 4C depicts a video image of another bodily position 430 of aparticipant on the pitcher's mound 422 which is indicative of a pitcher.In one approach, to detect a pitching motion, the video frames from acamera which is directed at the pitcher's mound can be processed andcompared to previously-determined templates of pitching motions. Inaddition to recognizing distinctive poses in the pitching motion, thechange in the pitcher's position on the field can indicate a pitch hasoccurred. In one approach, the system detects the pitcher first comingto rest, then his body making a distinctive move toward home plate. Asimilar kind of detection can be used to recognize when the pitcherthrows toward one of the bases in an attempt to pick off a base runner.

FIG. 5A depicts example locations of participants in the baseball parkof FIG. 1A, indicating potential merging of participants.

Two or more participants can be merged in a video frame when theirimages overlap, and one participant occludes another, at least in part.For example, one participant may stand in front of another participant,or move past another participant. Generally, merges can occur in imagespace or world space. A merge in image space may occur for a particularcamera angle in which blobs which represent the participants overlap inimage space, even if the two participants are not close physically. Inthis case, there may be another camera angle in which blobs whichrepresent the participants do not overlap in image space. A merge inworld space may occur when the participants are physically close to eachother so that blobs which represent the participants overlap in imagespace from all camera angles.

The merged participants may not be identifiable from the video frame.For example, in a time interval of video, a non-merged interval can bedefined in which there are one or more non-merged participants in thevideo frames, followed by a merged interval in which there are two ormore merged participants in the video frames, followed by a non-mergedinterval in which there are one or more non-merged participants in thevideo frames. The merged interval can be very brief, in the range of oneor more seconds, or longer, such as several seconds. If the participantsare not identifiable during the merged interval, this would result in agap in the digital record. The digital record can be used for variouspurposes including determining performance metrics, such as the time ittakes for the player to react to a hit, or to run to the ball, or to abase, for instance. The digital record can also be used to provide achronology of where each participant is throughout the game. The digitalrecord can also be used to cross-reference video clips/sequences tospecific participants. It is therefore desirable to provide as completea digital record as possible.

By observing successive locations of one or more participants (e.g., atleast first and second participants) before and/or after the mergedinterval, in the non-merged intervals, one can identify a role fromamong a plurality of roles in the baseball game which is consistent withthe successive locations. For example, this can be a role as aparticipant, such as one of the nine player roles, or a role as coach,such as one of the coach's roles. Based on the identified role, digitalrecord of the baseball game is updated to associate the role with thesuccessive locations of the one or more participants in the mergedinterval. In one approach, in the merged interval, we can associate theroles of the merged participants to one track. For instance, we can saythat, for a certain time segment we have a merged track which representsthe merged participants, e.g., the first baseman and the runner on firstbase. Separating the merged participants may or may not be possible. Inone approach, we attempt to split one merged blob into two or more humanfigures.

Splitting a blob containing multiple participants can occur temporallyusing template matching from one or more previous frames, and/orspatially within a single frame using blob shape and contours. In thetemporal approach, we find a previous frame where the participants canbe separately tracked, cut out one participant from this previous frame,and perform a correlation of the pixel intensity values from the oneparticipant to the pixel intensity values of the blob to distinguish theone participant in the blob.

In a spatial approach, we can analyze the area of the merged blob. Forexample, we can detect that a blob is too big to be one participant, andcan make a good guess as to whether it is two or three participant basedon its area. Also, based on the shape of the contours of the blob, wecan get an idea of how to split the blob. For example, if one large blobis formed by a thin connection between two larger areas, this could betwo players standing close to one another (e.g., first baseman and baserunner), and we know each larger area is a separate player. Also, thereare specific situations where players look merged but physically theyare not merged, such as when they appear to be vertically merged on thevideo screen, due to one player being in the foreground and one being inthe background, but vertically aligned. We can determine that twoparticipants are present due to the height of the blob beinginconsistent with one player. A blob detecting technique can determineif a blob meets at least one specified criterion, such as size, aspectratio, density or color profile. An erode filter can be used to reducethe size of a blob so that smaller features are removed, leaving one ormore central masses. This can be useful, e.g., when two participantswhich are connected by a field line are initially identified as oneblob. For instance, if the blob is too large, e.g., its height isgreater than the average height of a participant by a threshold amount,e.g., 25%, an erode filter is run on the blob. This is an example ofshape analysis which tries to break up a blob which is not likely torepresent a single participant.

Moreover, since the video frames are obtained from multiple cameras eachhaving different viewpoints of the baseball park, it is possible thatdifferent merged intervals are associated with different ones of themultiple cameras, or that a merged interval is associated with one ofthe multiple cameras, and no merged interval during the time interval isassociated with another of the multiple cameras. For example, consider ascenario where the coach walks to the pitcher's mound to talk to thepitcher. Successive participant locations 506 and 508 represent thecoach at successive points in time of a time interval, and participantlocation 328 represents the pitcher who is assumed to be essentiallystationary on the pitcher's mound during the time interval. From theviewpoint or perspective of the camera 500 along the left side of thebaseball park, the participants are not merged when the coach is at theparticipant location 508 (associated with a line of position 504), andthe participants are merged when the coach is at the participantlocation 506 (associated with a line of position 502).

As another example, consider a scenario where the first base coach isstationary at participant position 516, while the first baseman movesfrom participant position 324 to participant position 518. From theviewpoint or perspective of the camera 510 along the right side of thebaseball park, the participants are not merged when the first baseman isat the participant location 324 (associated with a line of position514), and the participants are merged when the first baseman is at theparticipant location 518 (associated with a line of position 512).Throughout the game, from each camera viewpoint, the players' images inthe video frames may be occasionally merged or separated.

FIG. 5B depicts a video image of a pitcher and a coach who are notmerged, from the viewpoint of the camera 500 of FIG. 5A. The pitcher520, on the pitcher's mound 522, is in the participant position 328 andthe coach 524 is in the participant position 508, to the right of thepitcher.

FIG. 5C depicts a video image of a pitcher and a coach who are merged,from the viewpoint of the camera 500 of FIG. 5A. The pitcher 520 isstill in the participant position 328, but the coach is now in theparticipant position 526, in front of the pitcher. This video imagerepresents what is seen by the video, where the human eye can discernthat there are two participants present. However, when the video framesare processed using image processing techniques to detect therepresentation of an object, such as one or more participants, in animage, it may not be possible to distinguish the two participants.

The image processing typically provides a rough outline image of theobject such as depicted by the image 528. FIG. 5D depicts an outlineimage 528 of the video image of FIG. 5C.

FIG. 5E1 depicts a timeline of the participants in FIG. 5E2 indicatingmerged and non-merged time intervals, from the perspective of the camera500 of FIG. 5A. Example time points t0-t18 are depicted, where a darksquare indicates a time point at which a merge occurs and a white squareindicates a time point at which a merge does not occur. Thus, t7-t10 isa merged interval in the time interval t0-t18, and t0-t6 and t11-t18 arenon-merged intervals. The same timeline applies to both of the mergedparticipants since, if one participant is merged, so is the other. Thenotation “tms” denotes the start of a merged interval and “tme” denotesthe end of the merged interval.

FIG. 5E2 depicts a portion of the baseball park of FIG. 5A, showingtracks of a pitcher and a coach in a time interval. The same scenario asin FIG. 5A is repeated, including the participant location 328 of thepitcher and the participant locations 506 and 508 of the coach. However,additional locations of the coach are also specified for each of thenineteen time points in FIG. 5E1. In particular, at t0, the coach is inthe dugout region 240 and begins walking toward the pitcher's moundregion 248. A dashed line circle represents an area 549 around thepitcher's mound region. Generally, an area can be defined aroundlandmarks such as the pitcher's mound and the bases, which can be usedto resolve merge scenarios. This area can typically be larger than theregions around the landmarks which are used to determine if aparticipant is at the landmark.

At t6, the coach is at participant location 542, within the area 549. Att7, the coach is at participant location 544, and is merged with thepitcher. At t8, the coach is at participant location 506, and is mergedwith the pitcher. At t9, the coach is at participant location 548, andis merged with the pitcher. At t10, the coach is at participant location550, and is merged with the pitcher. At t11, the coach is at participantlocation 508, and is no longer merged with the pitcher. The coachcontinues to moves back to the dugout region, where participant location552 corresponds to the final time point (tf). A track portion 543between participant locations 540 and 542 is a non-merged track of thecoach, while a track portion 546 between participant locations 544 and550 is a merged track (for both the coach and the pitcher) and a trackportion 547 between participant locations 508 and 552 is anothernon-merged track of the coach. Point 554 represents the track of thepitcher, and is half black and half white to represent merged andnon-merged intervals, respectively. The track portion 546 is a mergedtrack for both the coach and the pitcher because it cannot be associatedwith either the coach or the pitcher without analyzing the non-mergedtracks.

In this example, consider the participant who moves from the dugoutregion to the pitcher's mound region, and back to the dugout region, tobe a first, unknown participant, and the participant who remains on thepitcher's mound to be a second, unknown participant. One can identifythe role of the first participant from among a plurality of roles in thebaseball game which is consistent with the successive locations of thefirst and second participants in one or both of the non-mergedintervals. This can include determining that the first and secondparticipants are within a specified distance of the pitcher's mound ofthe playing field during at least a portion of the non-merged intervals,and identifying the role for the first participant as a coach and a rolefor the second participant as a pitcher, if, during the non-mergedinterval, the first participant moves from the pitcher's mound toward adugout while the second participant is at the pitcher's mound. That is,after the coach talks to the pitcher, the coach returns to the dugoutand the pitcher remains at the pitcher's mound. Thus, the track of thecoach in the non-merged interval can be used to identify both the coachand the pitcher during the merged interval.

FIG. 5E3 depicts a digital record for the locations of the pitcher inFIG. 5E2. Each entry includes one of the time points t0-t18, acorresponding location (which is a fixed location of the pitcher beingon the pitcher's mound, in this example), a corresponding region of theballpark, an indication of whether the track of the participant ismerged, an indication of whether the merge has been resolved, anindication of the time intervals used to resolve the merge, and anindication of the other merged participant. By identifying mergedtracks, it is possible to query the digital record to obtaincorresponding video clips for study, for instance, or to manuallyconfirm that a merge was correctly resolved. Resolving of a mergegenerally includes identifying the participants in the video frames inwhich a merge is detected. In some cases, a merge may not be resolvedand this may also be of interest. For example, it is possible to querythe digital record to obtain corresponding video clips of unresolvedmerges for study, for instance, to manually resolve a merge.

By identifying an indication of the time intervals used to resolve themerge, it is possible to query the digital record to obtaincorresponding video clips for study, for instance, to manually confirmthat a merge was correctly resolved. By identifying the other mergedparticipant, it is possible to query the digital record to obtaincorresponding video clips for study, for instance, to manually confirmthat the other merged participant was correctly identified.

The interval t7-t10 represents a portion of the respective track of theparticipant which is in the merged interval, and the intervals t0-t6 andt11-t18 represents portions of the respective track of the participantwhich are in non-merged intervals.

FIG. 5E4 depicts a digital record for the locations of the coach in FIG.5E2. Similar information is provided as in FIG. 5E3, but for the coach.Note that the locations of the coach, which define the track, vary asthe coach moves on the playing field, in the different regions,including the dugout region 240, then the left-side out-of-bounds region250, then the infield square or diamond 134, then the pitcher's moundregion 248, then back to the infield square or diamond 134, theleft-side out-of-bounds region 250, and the dugout region 240.

Note that a digital record or portion thereof may be created or updatedinitially without being associated with a particular participant untilthat participant is later identified.

Additional example merge scenarios are discussed next.

FIG. 6A depicts a video image of a first baseman and a base coach whoare not merged, from the viewpoint of the camera 510 of FIG. 5B. Theright-side foul line 114, first base 112 and the first base coach's box140 are depicted. The first base coach 602, in the first base coach'sbox 140, is in the participant position 516, and the first baseman is inthe participant position 324, to the right of the first base coach. Thefirst baseman is crouching in preparation for receiving a batted ball.Here, the participants are not merged. The first base coach 602 isstationary.

FIG. 6B depicts a video image of a first baseman and a base coach whoare merged, from the viewpoint of the camera 510 of FIG. 5B. Theparticipants become merged as the first baseman 604 moves to theparticipant position 518, behind the first base coach 602.

FIG. 6C depicts an outline image 606 of the video image of FIG. 6B.

FIG. 6D1 depicts a timeline of the participants in FIG. 6D2 indicatingmerged and non-merged time intervals, from the perspective of the camera510 of FIG. 5A. Example time points are t0-t7, where t2-t4 is a mergedinterval and t0-t1 and t5-t7 are non-merged intervals.

FIG. 6D2 depicts a portion of the baseball park of FIG. 5A, showingtracks of a first baseman and a base coach in a time interval,corresponding to the timeline of FIG. 6D1. A non-merged track 620represents the first baseman moving from the participant location 324 to622. A merged track 624 represents the first baseman moving to theparticipant locations 518, 626 and 628. The merged track 624 isassociated with both the first baseman and the first base coach becauseit cannot be associated with either the first baseman or the first basecoach without analyzing the non-merged tracks.

Another non-merged track 634 represents the first baseman moving fromthe participant location 630 to 632. A dashed line circle represents anarea 649 around the first base region which can be used to resolve mergescenarios. Point 516 represents the track of the first base coach, andis half black and half white to represent that merged and non-mergedintervals, respectively, occur.

FIG. 6D3 depicts a digital record for the first base coach in FIG. 6D2.Each entry includes one of the time points t0-t6, a correspondinglocation (which is a fixed location of the first base coach), acorresponding region of the ballpark (the first base coach's area 214),an indication of whether the track of the participant is merged, anindication of whether the merge has been resolved, an indication of thetime intervals used to resolve the merge, and an indication of the othermerged participant (the first baseman, or “1st bsmn”).

FIG. 6D4 depicts a digital record for the first baseman in FIG. 6D2. Thelocations of the first baseman, which define the track, vary as thefirst baseman moves on the playing field, in the different regions,including the first baseman's region 212, then the first base region218, and back to the first baseman's region 212. The other mergedparticipant is identified as the first base coach, or “1st bs co”).

FIG. 7A depicts further example locations of participants in thebaseball park of FIG. 1A, indicating potential merging of participants.As mentioned, the detection of merged participants in a video framedepends on the viewpoint of the camera. In this scenario, a base runneris running to first base. Participant locations 708 and 710 depict thebase runner, and participant location 706 depicts the first baseman, whois assumed to be stationary in the time interval under analysis. Forexample, the first baseman may be in position to catch the ball fromanother fielder. At the participant location 708, camera 510 has line ofposition 714 while a camera 700 located at the edge of center fieldincludes a line of position 704. At the participant location 710, camera510 has line of position 716 while the camera 700 has a line of position702. Generally, from the viewpoint of the camera 500, the first basemanwill be occluded by, and merged with, the base runner, when the baserunner is at first base. From the viewpoint of the camera 700, the baserunner will be occluded by, and merged with, the first baseman, when thebase runner is approaching first base.

FIG. 7B1 depicts a timeline of the participants in FIG. 7B2 indicatingmerged and non-merged time intervals, from the perspective of the camera510 of FIG. 7A. Example time points are t0-t18, where t6-t7 is a mergedinterval and t0-t5 and t8-t18 are non-merged intervals.

FIG. 7B2 depicts a portion of the baseball park of FIG. 7A, showingtracks of a first baseman and a first base runner in a time interval,corresponding to the timeline of FIG. 7B1. A non-merged track 736represents the first baseman moving from the participant location 324 to738. A merged track 744 represents the first baseman moving to theparticipant locations 740 and 706. The merged track 744 is associatedwith both the first baseman and the base runner because it cannot beassociated with either the first baseman or the base runner withoutanalyzing the non-merged tracks. Another non-merged track 746 representsthe first baseman moving to the participant location 742. Similarly, anon-merged track 724 represents the base runner moving from theparticipant location 720 to 722, 708 and 726. A merged track 745represents the base runner moving to the participant locations 710 and728. The merged track 745 is associated with both the first baseman andthe base runner because it cannot be associated with either the firstbaseman or the base runner without analyzing the non-merged tracks.Another non-merged track 732 represents the base runner moving from theparticipant location 730 to 734. This track could continue to the dugoutregion 238, for instance, such as when the base runner is called out atfirst, and returns to the dugout region.

FIG. 7B3 depicts a digital record for the first baseman in FIG. 7B2,from the perspective of camera 510. Each entry includes one of the timepoints t0418, a corresponding location, a corresponding region of theballpark (a first baseman's region 212 from t0-t4, and first base region218 from t5-t18), an indication of whether the track of the participantis merged, an indication of whether the merge has been resolved, anindication of the time intervals used to resolve the merge, and anindication of the other merged participant (the base runner or “bsrnr”).

FIG. 7B4 depicts a digital record for the first base runner in FIG. 7B2,from the perspective of camera 510. The entries indicate thecorresponding region of the ballpark (a first base runner's region 216from t0-t8 and a right-side out-of-bounds region 252 from t9-t18), andan indication of the other merged participant (the first baseman or “1stbs man”).

FIG. 7C1 depicts a timeline of the participants in FIG. 7C2 indicatingmerged and non-merged time intervals, from the perspective of the camera700 of FIG. 7A. A difference from FIG. 7B1 is that the merged intervalis t4-t6 instead of t6-t7 due to the different viewpoint of the camera700 compared to the camera 510. Also, t0-t3 and t7-t18 are non-mergedintervals.

FIG. 7C2 depicts a portion of the baseball park of FIG. 7A, showingtracks of a first baseman and a runner headed to first base in a timeinterval, corresponding to the timeline of FIG. 7C1. A non-merged track748 represents the first baseman moving from the participant location324 to 740. A merged track 750 represents the first baseman at theparticipant location 706. The merged track 750 is associated with boththe first baseman and the base runner because it cannot be associatedwith either the first baseman or the base runner without analyzing thenon-merged tracks. Another non-merged track 747 represents the firstbaseman at the participant location 742. Similarly, a non-merged track754 represents the base runner moving from the participant location 720to 722. A merged track 756 represents the base runner at the participantlocations 708, 726 and 710. The merged track 756 is associated with boththe first baseman and the base runner because it cannot be associatedwith either the first baseman or the base runner without analyzing thenon-merged tracks. Another non-merged track 752 represents the baserunner moving from the participant location 728 to 734.

FIG. 7C3 depicts a digital record for the first baseman in FIG. 7C2,from the perspective of camera 700. Each entry is the same as in FIG.7B3 except for the identification of the merged and non-mergedintervals.

FIG. 7C4 depicts a digital record for the first base runner in FIG. 7C2,from the perspective of camera 700. Each entry is the same as in FIG.7B4 except for the identification of the merged and non-mergedintervals.

FIG. 8A depicts a video image of a first baseman and a base runner whoare not merged, from the viewpoint of the camera 700 of FIG. 7A. Thefirst baseman 800, touching the base 112 with his foot, is in theparticipant position 706, and the base runner 802 is in the participantposition 708, to the right of the first baseman. The first baseman is ina position to catch a ball to force the base runner out. Here, theparticipants are not merged.

FIG. 8B depicts a video image of a first baseman and a base runner whoare merged, from the viewpoint of the camera 700 of FIG. 7A. Theparticipants become merged as the base runner 804 moves to theparticipant position 710, behind the first baseman 800.

FIG. 8C depicts an outline image 806 of the video image of FIG. 8B.

Similar images (not shown) can be obtained to represent the merged andnon-merged intervals from the viewpoint of the camera 510.

FIG. 9A depicts an example method for tracking baseball participantsfrom a video sequence, and updating a digital record. At step 900, oneor more cameras capture video frames. Step 902 analyzes the video framesto identify participants, such as by using image processing techniques.One example approach uses pattern recognition and morphologicalprocessing. Pattern recognition techniques include edge detection, whichinvolves detecting edges of objects in an image. The edges can bedetected based on contrasts in pixel data which extend generallyvertically or horizontally in an image or otherwise along a specificdirection. Edge detection can be performed using luminance and/orchrominance pixel data from an image. The chrominance pixel data canencompass any desired color space, including monochrome, RGB, YUV andHSV, among others, for instance. These edges may represent physicalfeatures of the object, such as the outline of a human.

The video images can be processed by analyzing the pixel data in eachimage. This processing can occur in real time as each image is received,or later on such as after the game. Real time processing can include,e.g., processing that occurs during a video broadcast of a live event.After edges are detected, morphological processing can be used to forman image of the one or more participants. The image data of the edges isrepresented as a binary image, in which each pixel is restricted to avalue of either 0 or 1. Each pixel in the background is displayed aswhite, while each pixel in the object is displayed as black. Binaryimages can be formed, e.g., by thresholding a grayscale image such thatpixels with a value greater than a threshold are set to 1, while pixelswith a value below the threshold are set to 0. The image data can beprocessed using two basic morphological operations, namely dilation anderosion. Further, one could repeat this process for different colorcomponents of the object. It is not necessary for each pixel to berestricted to a value of 0 or 1, or that it even be monochrome innature.

With erosion, every object pixel that is touching a background pixel ischanged into a background pixel. With dilation, every background pixelthat is touching an object pixel is changed into an object pixel.Erosion makes an object smaller, and can break a single object intomultiple objects. Dilation makes an object larger, and can mergemultiple objects into one. Opening is defined as an erosion followed bya dilation. Closing is the opposite operation and is defined as adilation followed by an erosion. Opening tends to removes small islandsand thin filaments of object pixels, while closing tends to removesislands and thin filaments of background pixels.

Another possible technique, a blob detecting, determines if a detectedobject meets at least one specified criterion, such as size, aspectratio, density or color profile. In some cases, an erode filter is usedto reduce the size of a blob so that smaller features are removed,leaving one or more central masses. This can be useful, e.g., when twoplayers which are connected by a field line are initially identified asone blob. For instance, if the blob is too large, e.g., its height isgreater than the average height of a player by a threshold amount, e.g.,25%, an erode filter is run on the blob. This is an example of shapeanalysis which tries to break up a blob which is not likely to representa single player. In some scenarios, the participants are still mergedafter such processing, in which case other techniques (step 914) can beused.

The motion of the participants can be modeled, e.g. using Kalmantracking, to follow each participant from frame to frame, and to handlecollisions (merging and splitting) by using the participants' velocitiesto associate the track before and after an overlap. We can adjust theKalman tracking based on knowledge of how the game proceeds. Some of theplayer's tracks can be marked as being merged tracks, unmerged tracks oruncertain tracks. After the merged participants clearly separate intotwo (or more) tracks, if we do not know which track is for whichparticipant, we can label those two new tracks as “uncertain.” Forexample, we might say that we have two tracks and we are uncertain whichone is for the left fielder and which one is for the center fielder.Thus, an uncertain track can be a type of unmerged track.

Step 902 can also automatically identify each participant based on wherethey are standing. This identification can be independent of theidentification of merged participants. For example, participants can beidentified based on their location in a particular region of the field.For example, a participant in the first baseman's region 212 (FIG. 2A)can be identified as the first baseman. Moreover, this identificationcan be made at specified points in the game, such, such as when a pitchis thrown.

Step 904 determines tracks of the participants based on their successivelocations at successive points in time. Once a participant is identifiedin a video frame, by identifying pixels in the video frames whichcorrespond to a human figure, the location of the participant in thebaseball park can be determined, e.g., using principles discussed inconnection with FIG. 1C. Sensor and registration data of the one or morecameras can be used. Since the orientation and position of the camerawith respect to the playing field is known, and the playing field can bemodeled as a flat surface, a set of pixels in a frame which correspondsto a human figure can be associated with a location on the playingfield.

Steps 906-912 relate to determining the start or end of a half-inning,while step 914 relates to identifying merged participants. Step 906determines movements of participants in across regions of the baseballpark, such as using the regions of FIG. 2B, and as discussed inconnection with FIGS. 3A-3B3. For example, this can indicate movement ofthe players between the playing field and the dugout. Step 908 includesdetermining bodily movements of one or more participants. For example, apitcher can be identified by observing a pitching motion, as discussedin connection with FIGS. 4A-4C. Step 910 includes determiningtrajectories of a ball. For example, this can include identifyingfielders warming up by playing catch, as discussed in connection withFIG. 4A. Step 912 includes detecting the start or end of a half-inningbased on one or more of the previous steps. In one approach, aprobability that a half-inning has started or ended can be associatedwith each step, and a final decision can be made by comparing a combinedprobability (which combines the probability of each step) to athreshold. In another approach, a final decision can be made based onone step or criterion. Step 916 includes updating one or more digitalrecords based on the previous steps.

Further details of steps 906, 908, 910 and 914 are provided next.

FIG. 9B depicts an example of further details of step 906 of FIG. 9A,for use in determining the start of a half-inning in step 912 of FIG.9A. At step 920, the tracks of the participants indicate that theparticipants have moved from the dugout region toward the playing fieldwithin a specified time. Optionally, one or more of the participants hasa specified minimum speed. At step 922, the tracks of the participantsindicate that the participants have moved from the dugout region to theoutfield within a specified time. Optionally, one or more of theparticipants has a specified minimum speed. At step 924, the tracks ofthe participants indicate that a participant has moved from the dugoutregion to a coach's box region. Step 926 determines that the start of ahalf-inning is about to occur based on one or more of the previoussteps.

FIG. 9C depicts an example of further details of step 908 of FIG. 9A,for use in determining the start of a half-inning in step 912 of FIG.9A. Step 930 determines that a bodily movement of a participant at apitcher's mound region corresponds to a pitching motion. Step 932determines that, at the same time, no participant is in the batter'sbox. Step 934 determines that the start of a half-inning is about tooccur based on one or more of the previous steps.

FIG. 9D depicts an example of further details of step 910 of FIG. 9A,for use in determining the start of a half-inning in step 912 of FIG.9A. Step 940 determines that the baseball travels back and forth betweenparticipants in a region of the playing field which is outside of theinfield square. Optionally, this occurs within a specified timeinterval. Step 942 determines that the start of a half-inning is aboutto occur based on the previous step.

FIG. 9E depicts an example of a process for determining the start of ahalf-inning Step 944 detects one or more events which indicate that thestart of a half-inning is about to occur, such as based on steps 920,922, 924, 930, 932, 940 and 942. Step 946 detects one or more eventswhich indicate that the half-inning is in progress, such as the firstpitch with the batter in the batter's box, or any other event whichtypically occurs during a half-inning and not during the time betweenhalf-innings. Step 948 updates a digital record to indicate that thestart of the half-inning has occurred. It is also possible to update thedigital record to indicate that the events of step 944 and/or 946 haveoccurred, including identifying the time and description of the event.

FIG. 9F depicts an example of further details of step 906 of FIG. 9A,for use in determining the end of a half-inning in step 912 of FIG. 9A.At step 950, the tracks of the participants indicate that theparticipants have moved toward the dugout region from the playing fieldwithin a specified time. Optionally, one or more of the participants hasa specified minimum speed. At step 952, the tracks of the participantsindicate that the participants have moved toward the dugout region fromthe outfield within a specified time. Optionally, one or more of theparticipants has a specified minimum speed. At step 954, the tracks ofthe participants indicate that a participant has moved to the dugoutregion from a coach's box region. Step 956 determines that the end of ahalf-inning has occurred based on one or more of the previous steps.

FIG. 9G depicts an example of further details of step 914 of FIG. 9A,for use in detecting merged participants. Step 960 identifies a timeinterval of a video sequence (e.g., a merged interval) in whichparticipants are merged. Step 962 identifies a time interval of thevideo sequence in which (e.g., a non-merged interval) in which theparticipants are not merged. Typically, the merged and non-mergedintervals are adjacent, although this is not required. Step 964identifies the participants by associating a role with one or more ofthe participant tracks in the non-merged interval. This can involve anumber of different scenarios.

For example, with first and second merged participants, the identifyingof the participants can include determining that the first and secondparticipants are within a specified distance of a base of the playingfield (e.g., within the circle 649 in FIG. 6D2, where the base is firstbase) during at least a portion of the non-merged interval, andidentifying the role for the first participant as a base coach of thebase, if, during the non-merged interval, the first participant is in acoach's box region associated with the base.

In another possible approach, the identifying of the participants caninclude determining that the first and second participants are within aspecified distance of a base of the playing field (e.g., within thecircle 649 in FIG. 6D2, where the base is first base) during at least aportion of the non-merged interval, and identifying the role for thefirst participant as a fielder assigned to the base (e.g., the firstbaseman) and a role for the second participant as a base runner to thebase, if, during the non-merged interval, the first participant is at afielder's location (e.g., the first baseman's region 212) associatedwith the base or at the base itself, and the second participant is atthe base or running away from the base, such as to the next base.

This can occur when the base runner is safe at the base and remains inthe base region. For first base, the base runner is allowed to overrunthe base and return to it if not called out. A longer time period can beset for determining if the base runner remains in the base region,compared to second and third base, where the base runner can be taggedout if he overruns the base.

In another possible approach, the identifying of the participants caninclude determining that the first and second participants are within aspecified distance of a base of the playing field (e.g., within thecircle 649 in FIG. 6D2, where the base is first base) during at least aportion of the non-merged interval, and identifying the role for thefirst participant as a fielder assigned to the base (e.g., the firstbaseman) and a role for the second participant as a base runner to thebase, if, during the non-merged interval, the first participant is at afielder's location (e.g., the first baseman's region 212) associatedwith the base while the second participant runs along a path to the base(e.g., the first base runner's region 216). Example base running pathsare discussed in connection with FIG. 9H.

FIG. 9H depicts the baseball park of FIG. 1A including straight andcurved base running paths. The path to first base, for instance (path970), is along the foul line region, in the first base runner's region216 (FIG. 2A). Respective base running paths from first to second base(116 or 971), from second base to third base (120 or 972), and fromthird base to home plate (974 or 973) can similarly be defined. Further,a relatively straight path (970, 116, 120 and 974) can be definedbetween bases where the base runner starts at one base and runs to thenext base (either stopping at the next base or continuing on to afurther base), and a relatively curved path (971, 972 and 973) can bedefined between bases where the base runner runs to one base, steppingon that base, then continues to the next base. An example of a straightpath (970) occurs when a player is at bat and runs to first base afterhitting a pitch, either stopping at first base or continuing on. Anotherexample of a straight path (116) occurs when a player is on first baseat the start of a play and then runs to second base when the player atbat hits a pitch, or when the player at first base attempts to stealsecond base. Similar examples of straight paths apply between second andthird base (path 120) and between third base and home plate (path 974).An example of a curved path occurs when a player at bat hits a pitch,runs to first base, rounds first base and continues to second base (path971). Similar examples of curved paths apply when the player starts atfirst and runs past second base, rounds second base, to third base orhome plate, or starts at second base, runs to third base, rounds thirdbase, and continues to home plate.

Referring still to FIG. 9G, in another possible approach, theidentifying of the participants can include determining that the firstand second participants are within a specified distance of a base of theplaying field (e.g., within the circle 649 in FIG. 6D2, where the baseis first base) during at least a portion of the non-merged interval, andidentifying the role for the first participant as a fielder assigned tothe base (e.g., the first baseman) and a role for the second participantas a base runner to the base, if, during the non-merged interval, thefirst participant is at a fielder's location associated with the baseand the second participant moves from a region of the base (e.g., region218 where the base is first base) to a dugout region of the baseballpark (e.g., via track 732 in FIG. 7B2). This can occur, e.g., when thebase runner is called out at the base and returns to the dugout.

In another possible approach, the identifying of the participants caninclude identifying the role for the first participant as a fielderassigned to one fielding position and a role for the second participantas a fielder assigned to another fielding position, if, during thenon-merged interval, the first participant is at a location associatedwith the one fielding position and the second participant is at alocation associated with the another fielding position. This can occur,e.g., when the left fielder and center fielder move close together tocatch a ball which is hit between them then return to their usuallocations after the play.

In another possible approach, the identifying of the participants caninclude determining that the first and second participants are within aspecified distance of a pitcher's mound (e.g., within the circle 549 inFIG. 5E2) of the playing field during at least a portion of thenon-merged interval, and identifying the role for the first participantas a coach and a role for the second participant as a pitcher, if,during the non-merged interval, the first participant moves from thepitcher's mound toward a dugout while the second participant is at thepitcher's mound. This can occur, e.g., when the coach talks to thepitcher, the returns to the dugout.

Other scenarios are possible as well which use knowledge of the rulesand customs of baseball to associate a participant's track with aparticular role in the game, to identify the participant andpotentially, one or more other participants.

The foregoing detailed description has been presented for purposes ofillustration and description. It is not intended to be exhaustive orlimited to the precise form disclosed. Many modifications and variationsare possible in light of the above teaching. The described embodimentswere chosen in order to best explain the principles of the technologyand its practical application, to thereby enable others skilled in theart to best utilize the technology in various embodiments and withvarious modifications as are suited to the particular use contemplated.It is intended that the scope of the technology be defined by the claimsappended hereto.

What is claimed is:
 1. Tangible, non-transitory computer readablestorage having computer readable software embodied thereon forprogramming at least one processor to perform a method for trackingparticipants in a baseball game at a baseball park, the baseball parkincludes a playing field, the method performed comprises: defining firstand second areas around a landmark on the playing field, the first andsecond areas each have a specified size and are positioned in specifiedway with respect to the landmark, and the second area is larger than thefirst area; defining a region of the baseball park, the region isoutside the first and second areas; analyzing video frames obtained fromone or more cameras, the video frames provide images of the participantsin the baseball park; based on the analyzing: determining a merged trackof at least first and second participants in the baseball park, themerged track indicates successive locations of the at least first andsecond participants in the baseball park during a merged time intervalin which the first and second participants are merged in the videoframes, determining one non-merged track of the first participant in thebaseball park during one non-merged time interval adjacent to the mergedtime interval in which the first and second participants are not mergedin the video frames, and determining one non-merged track of the secondparticipant in the baseball park during the one non-merged timeinterval; determining that the first participant remains within thefirst area during the one non-merged time interval based on the onenon-merged track of the first participant; determining that the secondparticipant performs a movement during the one non-merged time based onthe one non-merged track of the second participant, the movement is atleast one of: from the region to the second area or from the second areato the region; identifying a role of the first participant from among aplurality of roles in the baseball game, the identifying the role of thefirst participant comprises determining that the first participant isassociated with the landmark, the determining that the first participantis associated with the landmark is based on the determining that thefirst participant remains within the first area during the onenon-merged time interval; identifying a role of the second participantfrom among the plurality of roles, the identifying the role of thesecond participant comprises determining that the second participant isassociated with the region, the determining that the second participantis associated with the region is based on the determining that thesecond participant performs the movement during the one non-merged timeinterval; and responsive to the identifying the role of the firstparticipant and the identifying the role of the second participant,updating a digital record of the baseball game to associate the firstparticipant with the merged track and the one non-merged track of thefirst participant, and to indicate that the first participant was mergedwith the second participant in the merged track.
 2. The tangible,non-transitory computer readable storage of claim 1, wherein the methodperformed further comprises: responsive to the identifying the role ofthe first participant and the identifying the role of the secondparticipant, updating the digital record to associate the secondparticipant with the merged track and with the one non-merged track ofthe second participant and to indicate that the second participant wasmerged with the first participant in the merged track.
 3. The tangible,non-transitory computer readable storage of claim 1, wherein: theidentifying of the role of the second participant is based on theidentifying the role of the first participant.
 4. The tangible,non-transitory computer readable storage of claim 1, wherein: thelandmark comprises a base; the role of the first participant is abaseman of the base; and the role of the second participant is a baserunner of the base.
 5. The tangible, non-transitory computer readablestorage of claim 1, wherein: the landmark comprises a base; the role ofthe first participant is a base runner of the base; and the role of thesecond participant is a coach.
 6. The tangible, non-transitory computerreadable storage of claim 1, wherein: the landmark comprises a pitcher'smound; the role of the first participant is a pitcher; the role of thesecond participant is a coach; the first area is around the pitcher'smound; the second area is around the first area; and the regioncomprises a dugout.
 7. The tangible, non-transitory computer readablestorage of claim 1, wherein: the landmark comprises at least one of anon-deck circle, a coach's box, a catcher's box or a batter's box.
 8. Amethod for tracking participants in a baseball game at a baseball park,the baseball park includes a playing field, the method comprising theprocessor-implemented steps of: analyzing a plurality of successivevideo frames obtained from one or more cameras, the video frames provideimages of the participants in the baseball park; based on the analyzing:determining, among the plurality of successive video frames, videoframes comprising a representation of a first participant merged with arepresentation of a second participant in the baseball park during amerged time interval, video frames comprising a representation of thefirst participant not merged with another participant in a non-mergedtime interval, and video frames comprising a representation of thesecond participant not merged with another participant in the non-mergedtime interval; determining a merged track in the baseball park of thefirst and second participants based on the video frames comprising therepresentation of the first participant merged with the representationof the second participant; determining a non-merged track in thebaseball park of the first participant based on the video framescomprising the representation of the first participant not merged withanother participant; determining that the non-merged track of the firstparticipant remains within a first area around a landmark on the playingfield; determining a non-merged track in the baseball park of the secondparticipant based on the video frames comprising the representation ofthe second participant not merged with another participant; determiningthat the non-merged track of the second participant indicates a movementof the second participant, the movement is at least one of: from asecond area around the landmark to a region of the baseball park outsidethe second area, the second area is larger than the first area, or fromthe region to the second area; based on the determining that thenon-merged track of the first participant remains within the first area,identifying a role of the first participant in the baseball game asbeing associated with the landmark and updating a digital recordassociated with the role of the first participant with locations of thenon-merged track of the first participant cross-referenced to thenon-merged time interval and to an indication that the first participantis not merged with another participant during the non-merged timeinterval; and based on the determining that the non-merged track of thesecond participant indicates the movement of the second participant,identifying a role of the second participant in the baseball game asbeing associated with the region and updating a digital recordassociated with the role of the second participant with locations of thenon-merged track of the second participant cross-referenced to thenon-merged time interval and to an indication that the secondparticipant is not merged with another participant during the non-mergedtime interval.
 9. The method of claim 8, further comprising: updatingthe digital record associated with the role of the second participantwith the locations of the merged track cross-referenced to: the mergedtime interval, an indication that a merger involving the secondparticipant has been resolved and an indication that the mergerinvolving the second participant was with the first participant.
 10. Themethod of claim 8, wherein: the identifying the role of the secondparticipant is based on the role of the first participant; and theidentifying the role of the first participant is based on the role ofthe second participant.
 11. The method of claim 8, further comprising:updating the digital record associated with the role of the firstparticipant to identify a time interval of the non-merged time intervalwhich was used to identify the role of the first participant.
 12. Themethod of claim 8, wherein: the landmark comprises a base; the regioncomprises a coach's box region associated with the base; the role of thefirst participant is a baseman of the base; and the role of the secondparticipant is a base coach of the base.
 13. The method of claim 8,further comprising: updating the digital record associated with the roleof the first participant to identify a time interval of the non-mergedtime interval which was used to resolve the merged track.
 14. The methodof claim 8, wherein: the landmark comprises a base; the role of thefirst participant is a baseman of the base; and the role of the secondparticipant is a base runner.
 15. The method of claim 8, wherein: thelandmark comprises a pitcher's mound; the region comprises a dugout; therole of the first participant is a pitcher; and the role of the secondparticipant is a coach.
 16. The method of claim 8, wherein: the landmarkcomprises a base; the role of the first participant is a baseman of thebase; the region comprises a dugout; and the role of the secondparticipant is a base runner.
 17. The tangible, non-transitory computerreadable storage of claim 1, wherein the method performed furthercomprises: responsive to the identifying the role of the firstparticipant and the identifying the role of the second participant,updating the digital record to indicate a time interval of the onenon-merged time interval of the second participant used to resolve themerged track.
 18. The tangible, non-transitory computer readable storageof claim 1, wherein the method performed further comprises: based on theanalyzing: determining another non-merged track of the first participantin the baseball park during another non-merged time interval adjacent tothe merged time interval in which the first and second participants arenot merged, and determining another non-merged track of the secondparticipant in the baseball park during the another non-merged timeinterval; determining that the first participant remains within thefirst area during the another non-merged time interval based on theanother non-merged track of the first participant; determining that thesecond participant performs a movement from the region to the secondarea based on the one non-merged track of the second participant; anddetermining that the second participant performs a movement from thesecond area to the region based on the another non-merged track of thesecond participant; wherein: the determining that the first participantis associated with the landmark is based on the determining that thefirst participant remains within the first area during the anothernon-merged time interval; and the determining that the secondparticipant is associated with the region is based on the determiningthat the second participant performs the movement from the region to thesecond area and the determining that the second participant performs themovement from the second area to the region.
 19. An apparatus fortracking participants in a game at a playing field, the apparatuscomprising: a processor readable storage device having processorreadable code embodied thereon; and a processor, the processor isconfigured to execute the processor readable code to: analyze aplurality of successive video frames obtained from one or more cameras,the video frames provide images of the participants in the game; basedon the analyzing: determine, among the plurality of successive videoframes, video frames comprising a representation of a first participantmerged with a representation of a second participant in the game duringa merged time interval, video frames comprising a representation of thefirst participant not merged with another participant in a non-mergedtime interval, and video frames comprising a representation of thesecond participant not merged with another participant in the non-mergedtime interval; determine a merged track in the game of the first andsecond participants based on the video frames comprising therepresentation of the first participant merged with the representationof the second participant; determine a non-merged track in the game ofthe first participant based on the video frames comprising therepresentation of the first participant not merged with anotherparticipant; determine that the non-merged track of the firstparticipant remains within a first area around a landmark on the playingfield; determine a non-merged track in the game of the secondparticipant based on the video frames comprising the representation ofthe second participant not merged with another participant; determinethat the non-merged track of the second participant indicates a movementof the second participant, the movement is at least one of: from asecond area around the landmark to a region of the game outside thesecond area, the second area is larger than the first area, or from theregion to the second area; based on the determining that the non-mergedtrack of the first participant remains within the first area, identify arole of the first participant in the game as being associated with thelandmark and updating a digital record associated with the role of thefirst participant with locations of the non-merged track of the firstparticipant cross-referenced to the non-merged time interval and to anindication that the first participant is not merged with anotherparticipant during the non-merged time interval; and based on thedetermining that the non-merged track of the second participantindicates the movement of the second participant, identify a role of thesecond participant in the game as being associated with the region andupdating a digital record associated with the role of the secondparticipant with locations of the non-merged track of the secondparticipant cross-referenced to the non-merged time interval and to anindication that the second participant is not merged with anotherparticipant during the non-merged time interval.
 20. The apparatus ofclaim 19, wherein the processor is configured to execute the processorreadable code to: update the digital record associated with the role ofthe second participant with the locations of the merged trackcross-referenced to: the merged time interval, an indication that amerger involving the second participant has been resolved and anindication that the merger involving the second participant was with thefirst participant.
 21. The apparatus of claim 19, wherein: theidentifying the role of the second participant is based on the role ofthe first participant; and the identifying the role of the firstparticipant is based on the role of the second participant.
 22. Theapparatus of claim 19, wherein the processor is configured to executethe processor readable code to: update the digital record associatedwith the role of the first participant to identify a time interval ofthe non-merged time interval which was used to identify the role of thefirst participant.
 23. The apparatus of claim 19, wherein the processoris configured to execute the processor readable code to: update thedigital record associated with the role of the first participant toidentify a time interval of the non-merged time interval which was usedto resolve the merged track.
 24. The apparatus of claim 19, wherein: thegame is a baseball game.
 25. The apparatus of claim 24, wherein: thelandmark comprises a base; the region comprises a coach's box regionassociated with the base; the role of the first participant is a basemanof the base; and the role of the second participant is a base coach ofthe base.
 26. The apparatus of claim 24, wherein: the landmark comprisesa base; the role of the first participant is a baseman of the base; andthe role of the second participant is a base runner.
 27. The apparatusof claim 24, wherein: the landmark comprises a pitcher's mound; theregion comprises a dugout; the role of the first participant is apitcher; and the role of the second participant is a coach.
 28. Theapparatus of claim 24, wherein: the landmark comprises a base; the roleof the first participant is a baseman of the base; the region comprisesa dugout; and the role of the second participant is a base runner.